ChronoLLM: Using LLMs to Generate Chrono Digital Twins and to Judge Their Suitability for Simulation
Please login to view abstract download link
We present ChronoLLM, a framework that leverages (i) synthesized data from small datasets, and (ii) customized large language models (LLMs), to produce Chrono Digital Twins (DTs) for multi-physics simulations [1]. By fine-tuning open-source LLMs for code generation within the PyChrono environment [2], ChronoLLM automates and improves the creation of simulation scripts for multibody dynamics, finite element analysis, vehicle dynamics, robotic dynamics, and sensor simulations. This integration accelerates simulation setup and enhances the accuracy of generated simulation code. To overcome the challenges posed by small datasets, ChronoLLM employs data synthesis techniques to expand and enrich the training data, enabling the customized LLMs to learn effectively despite data scarcity. This approach, which relies on prompt engineering and fine-tuning, improves the model’s pre-trained semantic understanding and enhances its ability to generate high-quality PyChrono simulation scripts. To evaluate the proficiency of ChronoLLM, we employ SimBench, a benchmark designed to assess the ability of various student LLMs (S-LLMs) to generate high-quality DTs for Chrono simulation. SimBench utilizes a rule-based judge LLM (J-LLM) that combines predefined rules with human-in-the-loop guidance to assign scores to the DTs produced by S-LLMs, ensuring a consistent, objective, and expert-inspired evaluation protocol. The J-LLM is thus enabling a quantifiable assessment of ChronoLLM’s DT-generation capabilities. The benchmarking results reported demonstrate improvements in simulation-setup speed, code accuracy, and computational efficiency when using ChronoLLM. The framework expedites the development and testing of complex DTs and provides a scalable, AI-enhanced approach that can be used for Digital Twin generation by other CAE packages such as ABAQUS, ANSYS, ADAMS, etc. For code and data see [3], and the associated GitHub Repository.
